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Method, device and arrangement for inserting extra information 



The invention relates to a method of inserting extra information in an input 
signal in a first domain. 

The invention further relates to a device for inserting extra information in an 
input signal in a first domain. 
5 The invention further relates, to an arrangement for inserting extra information 

in an input signal in a first domain. 

The invention further relates to a computer program product. 



1 o Watermarking, the process of inserting extra information in an input signal 

such as audio or video, is an important and well-known technique to mark or protect those 
input signals. A movie can be watermarked so its origin can be identified, or unauthorized 
copies can be distinguished from the original. Watermarks can be used with still images to 
locate copies reproduced by unauthorized third parties, by simply downloading images from 

15 the information services offered by those third parties and examining the downloaded images 
for the watermark. 

Watermarks can also be used to embed metadata, such as an Internet Uniform 
Resource Locator (URL), in the input signal, for instance in a movie. A user who views the 
movie at his personal entertainment station can access the embedded metadata to access, for 
20 instance, the World-Wide Web site of the movie. 

In known watermarking schemes, the input signal that has to be watermarked 
is first converted from a first domain, e.g. the audio domain, to a second domain, e.g. the 
electrical domain. The representation in the second domain of the input signal is then 
analyzed and watermarked. The watermark is then inserted into the input signal, and the 
25 thusly obtained watermarked signal can subsequently be output to the first domain. When the 
watermarked signal is distributed further, it can be analyzed to retrieve the watermark in 
order to prove ownership, retrieve embedded metadata, and so on. 

A disadvantage of these known schemes is that the watermarked signal exists 
independently of the original, unmodified, input signal. The input signal needs to be fed to a 
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device which generates the watermark and outputs the watermarked signal, but this does not 
affect the original input signal. Since it is undesirable to make the original unwatermarked 
signal available, the known schemes can only be used to watermark signals in controlled 
situations, such as in a recording studio. The music generated by a band during a live 
6t unplugged" performance, for example, is released in an uncontrolled situation, and bootleg 
copies can be made without the watermark. This disadvantage becomes even more apparent 
with visual input signals, such as exhibitions or interviews. An interview in a public location 
cannot be completely protected, since it is always possible for a third party to make his own 
recording of the interview with a device that does not embed the watermark in the recording. 



It is an object of the invention to provide a method according to the preamble, 
which produces a watermarked input signal and prevents the availability of unwatermarked 
specimens of the input signal, even in uncontrolled environments. 

1 5 This object is achieved in the invention in a method which comprises the steps 

of receiving a representation of the input signal, the representation being in a second domain 
different from the first domain, generating a watermark comprising the extra information, and 
outputting a watermark signal representing the watermark to the first domain for merging the 
watermark signal with the input signal. Preferably the first domain is one of the acoustic 

20 domain and the visual domain, and the second domain is the electrical domain. The method 
according to the invention generates a watermark for the representation of the input signal, 
and instead of inserting the watermark in the representation and outputting that representation 
as a watermarked signal, it outputs a representation of the watermark to the first domain 
where it can merge with the original input signal. Thus, the original input signal itself is 

25 watermarked, and no unwatermarked specimens exist. For instance, consider the exhibition 
of a painting. A watermark is generated for the painting, and instead of outputting a copy of 
the painting with the watermark embedded, a visual watermark is projected on the actual 
painting hanging on the wall. Now all photographs etc. made of the painting will have the 
watermark embedded. 

30 In an embodiment the watermark is generated further based on the input 

signal. An advantage of this embodiment is that adapting the watermark based on the input 
signal allows the creation of a watermark that better fits the input signal, for example by 
being less obtrusive. To this end, the watermark generation may comprise perceptual 
masking. For example, in the case of an audio input signal from an interview, the voice of the 
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person being interviewed could be analyzed and the watermark signal would be adapted 
accordingly. Similarly, in the case of a video input signal from an exhibited piece of art, the 
lighting conditions could be analyzed and the watermark signal would be adapted 
accordingly. 

5 In a further embodiment the watermark signal is output towards a source of the 

input signal. An advantage of this embodiment is that in this way, multiple input signals are 
mixed with the watermark signal, regardless of their respective directions. In the case of the 
exhibited piece of art, the watermark signal could be projected by a lighting system on the 
piece of art so that there is no possibility that a video signal can be generated without 

1 0 including the watermark signal mixed with the image of the piece of art. 

In a further embodiment the watermark signal is output on a surface of a 
source of the input signal. An advantage of this embodiment is that it allows the source of the 
input signal to protect itself against unauthorized recordings. For example, a person wearing 
a piece of clothing comprising a flexible display could project a visual watermark signal from 

15 the flexible display so that anyone who takes a photo of him will always have a visual 
watermark in the photo. 

It is a further object of the invention to provide a device according to the 
preamble, which produces a watermarked input signal and prevents the availability of 
unwatermarked specimens of the input signal, even in uncontrolled environments. 

20 This object is achieved in the invention in a device which comprises receiving 

means for receiving a representation of the input signal, the representation being in a second 
domain different from the first domain, generating means for generating a watermark 
comprising the extra information, and output means for outputting a watermark signal 
representing the watermark to the first domain for merging the watermark signal with the 

25 input signal. 

It is a further object of the invention to provide an arrangement according to 
the preamble, which produces a watermarked input signal and prevents the availability of 
unwatermarked specimens of the input signal, even in uncontrolled environments. 

This object is achieved in the invention in an arrangement which comprises 
30 receiving means for receiving a representation of the input signal, the representation being in 
a second domain different from the first domain, generating means for generating a 
watermark comprising the extra information, output means for outputting a watermark signal 
representing the watermark to the first domain for merging the watermark signal with the 
input signal, and merging means for generating a watermarked input signal by receiving the 
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merged watermark signal and input signal. In this arrangement according to the invention, the 
watermark signal and the original input signal are mixed to obtain a watermarked input 
signal, which can then be processed further using other techniques, if desired. The 
arrangement is typically realized using a first device which is arranged to receive the 
representation, generate the watermark and output the watermark signal as described above, 
and a second device which obtains the original input signal mixed with the watermark signal. 
In the case of the exhibition of the piece of art, the first device would be the lighting system 
projecting the watermark signal onto the piece of art, and the second device would be, for 
instance, a camera. 

It is a further object of the invention to provide a computer program product 
being arranged for causing a processor to execute the method according to the invention. 

These and other aspects of the invention will be apparent from and elucidated 
with reference to the embodiments shown in the drawing, in which: 

Figure 1 schematically shows a first embodiment of the device according to 

the invention; 

Figure 2 schematically shows a second embodiment of the device; and 
Figure 3 schematically shows a third embodiment of the device. 

Throughout the figures, same reference numerals indicate similar or 
corresponding features. Some of the features indicated in the drawings are typically 
implemented in software, and as such represent software entities, such as software modules 
or objects. 

Figure 1 schematically shows a first embodiment of the device 100 according 
to the invention. The device 100 comprises receiving module 101, generating module 102, 
and output module 103. A source 110 generates an input signal 111. The present invention 
can be used in many situations, with many different sources 110 generating many different 
input signals 111. For instance, the source 110 can be a natural person giving an interview, or 
singing a song or giving a lecture. The source 110 can be a band, which gives a concert, or 
another live performance. The input signal 1 1 1 is in a first domain, often the acoustical or 
visual domain, e.g. in the form of an audio or video signal, although other domains are also 
possible. 
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The receiving module 101 receives a representation of the input signal 111. 
This representation is in a second domain different from the first domain. Preferably the 
second domain is the electrical domain, in which case the representation typically is an 
electrical signal obtained by having a microphone record the input signal 111. Other 
5 conversions are also possible, for instance by digitizing the input signal 1 1 1 or receiving a 
broadcast signal from the source 110. The reception of the representation may optionally 
involve further processing steps after picking up the input signal and converting it to the 
second domain. 

The representation is fed to the generating module 102, which generates a 

1 0 watermark comprising extra information 1 12 for use with the representation. The watermark 
can be generated using any watermarking technique appropriate for the input signal 111 and 
is preferably based on the representation. In particular, additive watermarking techniques can 
be used. Using additive watermarking techniques, a watermark is generated which is 
normally mixed with the input signal 1 1 1 by means of addition to form a watermarked signal. 

15 However, in accordance with the invention, the output module 103 outputs a watermark 

signal 113 representing the watermark to the first domain for merging the watermark signal 
113 with the input signal 111. 

The outputting can be done directly or indirectly. In the direct case, the 
watermark signal 113 can be output so that it directly mixes with the input signal 111 and 

20 they reach some external reception unit 114 together. In the indirect case, the watermark 
signal 113 reaches the reception unit 1 14 via the input signal 111 that needs to be 
watermarked. This can be done, for instance, by outputting the watermark signal 113 towards 
the source 1 10 of the input signal 111. This is particularly appropriate when the first domain 
is the visual domain. For instance, consider the exhibition of a painting. A watermark is 

25 generated for the painting, and instead of outputting a copy of the painting with the 

watermark embedded, a visual watermark is projected on the actual painting hanging on the 
wall. Now all photographs etc. made of the painting will have the watermark embedded. 

The watermark being generated can be imperceptible or unobtrusive. 
Imperceptible watermarks are invisible or inaudible for a human observer, while unobtrusive 

30 watermarks are visible or audible but not annoying to the human observer. The choice 

depends on the particular application of the device 100. It may be desirable in some situations 
to reveal the fact that the input signal 1 1 1 is being watermarked, so that potential adversaries 
are discouraged from producing an illicit recording. The watermark can alternatively be used 
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to embed metadata in the input signal, which typically is not intended for human observers, 

so it should be invisible and inaudible. 

The extra information 1 12 can be static, e.g. constant over time, or dynamic. 

The identity of the person interviewed is an example of a static payload, while information 
5 about the time and location of the interview is an example of a dynamic payload, since it may 

differ during the interview. 

The device 100 may optionally generate the watermark further based on an 

analysis of environmental parameters of the input signal 111. This allows the creation of a 

watermark that better fits the input signal 1 1 1, for example by being less obtrusive. To this 
10 end, the generating module 102 may be arranged for perceptual masking. For example, in the 

case of an audio input signal from an interview, the voice of the person being interviewed 

could be analyzed and the watermark signal would be adapted accordingly. Similarly, in the 

case of a video input signal from an exhibited piece of art, the lighting conditions could be 

analyzed and the watermark signal would be adapted accordingly. These adaptations are 
1 5 made in order to improve the perceptive performance of the device 100. 

The arrangement comprising the receiving module 101, generating module 

102, output module 103 and reception unit 1 14 is capable of generating a watermarked input 

signal 115 from the original input signal 111. Using this arrangement, no unwatermarked 

specimens of the input signal 111 are made available. 
20 The device 100 can be realized as a computer program product 120 being 

arranged for causing a processor to execute the method according to the invention. This 

computer program product 120 enables a programmable device when executing said 

computer program product to function as the device 100. 

Figure 2 schematically shows a second embodiment of the device 100, here 
25 shown as a small portable unit, which can be worn or carried by a person 200. This allows 

that person 200, for instance, to watermark anything she says by having her portable unit 

emit an acoustic watermark signal 113 that mixes with her speech. 

The device 100 can also be realized as a flexible display or loudspeaker 

integrated in a piece of clothing. This allows the wearer of that piece of clothing to 
30 watermark anything she says, or to have any picture taken of her to be watermarked. In that 

case, the watermark signal 1 13 is output on a surface of the source 200 of the input signal 

111. 

The method according to the invention is especially suitable with mobile 
devices, such as a mobile telephone, as the reception unit 114. Since the watermarked input 
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signal 115, preferably being an audio signal, travels over the acoustical domain to the 
reception unit 1 14, it follows that any device having a microphone or similar reception means 
can pick it up, and, with suitable equipment, can detect the watermark and process it. 

In practice, this means that the owner of a mobile device comprising such 
5 reception means can detect this watermark by simply pointing his mobile device towards the 
output module 103. 

Figure 3 schematically shows a third embodiment of the device 100, shown 
here with a lighting unit 300 as part of the output module 103. The watermark signal 1 13 is 
output towards the source 1 1 0 of the input signal 111, which can be any source 110 that 

10 needs to be watermarked, such as a piece of art or a soccer match or some other event. Light 
filters in the lighting unit 300 containing the watermark pattern can be used to create the 
required lighting. This "watermarked light" is then used to lighten the scene that may be 
photographed or filmed by third parties. The light filters may contain a dynamically changing 
watermark pattern to better protect against video recordings. 

15 The device 100 may be arranged for simultaneous generation of audio and 

video watermarks. It is possible to generate and insert a watermark in the conventional way 
in the input signal 111 mixed with the watermark signal 113 produced by the device 100. 
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1 . A method of inserting extra information in an input signal (1 1 1) in a first 
domain, comprising the steps of 

• receiving a representation of the input signal (1 1 1), the representation being in a second 
domain different from the first domain, 

5 • generating a watermark comprising the extra information, and 

• outputting a watermark signal (113) representing the watermark to the first domain for 
merging the watermark signal (1 13) with the input signal (1 1 1). 

2. A method as claimed in claim 1, where the first domain is one of the acoustic 
1 0 domain and the visual domain. 

3. A method as claimed in claim 1 or 2, where the second domain is the electrical 
domain. 

15 4. A method as claimed in claim 1 , where the watermark is generated based on 

the input signal (1 1 1). 

5. A method as claimed in claim 1, where the watermark signal is output towards 
a source (1 10) of the input signal (1 1 1). 

20 

6. A method as claimed in claim 1, where the watermark signal is output on a 
surface of a source (1 10) of the input signal (1 1 1). 

7. A device (100) for inserting extra information in an input signal (1 1 1) in a first 
25 domain, comprising 

• receiving means (101) for receiving a representation of the input signal (1 1 1), the 
representation being in a second domain different from the first domain, 

• generating means (102) for generating a watermark comprising the extra information, and 
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• output means (1 03) for outputting a watermark signal (113) representing the watermark to 
the first domain for merging the watermark signal (113) with the input signal (1 1 1). 

8. An arrangement for inserting extra information in an input signal (1 1 1) in a 
5 first domain, comprising 

• receiving means (101) for receiving a representation of the input signal (1 1 1), the 
representation being in a second domain different from the first domain, 

• generating means (1 02) for generating a watermark comprising the extra information, and 

• output means (103) for outputting a watermark signal (1 1 3) representing the watermark to 
10 the first domain for merging the watermark signal (113) with the input signal (111), and 

• merging means (1 14) for generating a watermarked input signal (115) by receiving the 
merged watermark signal (113) and input signal (1 1 1). 

9. A computer program product (120) being arranged for causing a processor to 
1 5 execute the method of claim 1 . 
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